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PREFACE 


This  report  presents  the  provincial  results  of  the  English  33  Diploma 
Examination  administered  on  June  14  (Part  A)  and  June  25  (Part  B)  1984. 

During  this  administration,  the  English  33  Diploma  Examination  was  written  by 
6074  students.  In  addition  to  the  examination  results,  this  report  provides 
information  about  the  examination  development  process  and  the  examination 
itself. 
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CHAPTER  1 


THE  GRADE  12  DIPLOMA  EXAMINATIONS  PROGRAM 


Introduction 

The  Grade  12  Diploma  Examinations  Program,  which  is  an  integral  part  of  the 
high  school  diploma  requirements,  is  intended  to  develop  and  maintain 
excellence  in  educational  standards  through  certification  of  academic 
achievement.  The  program  consists  of  course-specific  examinations  that  are 
based  on  the  prescribed  Program  of  Studies  for  Senior  High  Schools  for  the 
following  Grade  12  courses:  English  30,  English  33,  Social  Studies  30, 

Mathematics  30,  Biology  30,  Chemistry  30,  and  Physics  30. 

All  Grade  12  students  in  Alberta  are  now  required  to  write  at  least  ONE 
diploma  examination  in  order  to  receive  a high  school  diploma. 

Alberta  Education  issues  two  distinct  high  school  diplomas:  the  General  High 

School  Diploma  and  the  Advanced  High  School  Diploma. 


General  High  School  Diploma 

To  earn  a General  High  School  Diploma,  a student  must  obtain  course  credit  in 
either  English  30  or  English  33,  and  obtain  100  credits  distributed  over 
courses  as  specified  in  the  Junior-Senior  High  School  Handbook.  Some 
students  who  are  working  toward  the  general  diploma  may  wish  to  obtain  credits 
in  other  diploma  examination  courses  (i.e..  Social  Studies  30,  Mathematics  30, 
Biology  30,  Chemistry  30,  and  Physics  30).  To  obtain  credits  in  these 
courses,  a student  must  also  write  the  appropriate  diploma  examination 
regardless  of  which  type  of  diploma  he  wishes  to  receive. 


Advanced  High  School  Diploma 

The  Advanced  High  School  Diploma  represents  achievement  in  an  academic  program 
that  includes  language  arts  (English),  social  studies,  mathematics,  and 
science.  To  earn  an  Advanced  High  School  Diploma,  a student  must  satisfy  the 
current  course  and  credit  requirements  for  a General  High  School  Diploma  and 
obtain  course  credit  in  English  30,  Social  Studies  30,  Mathematics  30,  and  ONE 
of  Biology  30,  Chemistry  30,  or  Physics  30. 


Awarding  of  Course  Credits 

To  obtain  credit  in  any  Grade  12  (30-level)  course,  a student  must  earn  a 
final  mark  of  50 % or  better.  To  obtain  credit  in  a Grade  12  (30-level) 
diploma  examination  course,  a student  must  write  the  appropriate  diploma 
examination  and  attain  a final  blended  mark  of  50%  or  better.  The  "final 
blended  mark"  is  made  up  of  50%  of  the  mark  awarded  by  the  school  and  50%  of 
the  diploma  examination  mark.  For  example,  a student  taking  English  33  might 
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have  a mark  of  45%  from  his  school  and  a mark  of  57%  on  the  diploma 
examination.  This  student's  final  mark  is  the  average  of  the  two  marks,  or 
51%.  He  has  therefore  earned  credits  in  English  33  because  his  final  mark  is 
over  50%. 


Transitional  Provisions 

Alberta  Education  recognizes  all  course  credits  earned  prior  to  September  1, 
1983,  for  the  purpose  of  awarding  the  General  High  School  Diploma. 

Students  who  have  completed  partial  requirements  for  the  Advanced  High  School 
Diploma  prior  to  September  1,  1983,  and  who  are  enrolled  in  Grade  12  courses, 
may  apply  any  of  the  previously  completed  required  diploma  examination 
subjects  toward  a diploma  provided  that  they  have  earned  a final  course  mark 
of  50%  or  better  in  each  subject. 


Award  of  Excellence 

When  a candidate  for  an  Advanced  High  School  Diploma  obtains  a final  average 
of  80%  or  higher  on  the  four  required  diploma  examination  courses  with  not 
less  than  65%  in  any  one  of  these  four  required  courses,  he  receives  an  Award 
of  Excellence.  This  Award  of  Excellence  is  noted  on  the  student's  Advanced 
High  School  Diploma. 

When  a student  writes  two  or  three  of  the  diploma  examinations  in  the  sciences 
(i.e..  Biology  30,  Chemistry  30,  and  Physics  30),  the  highest  of  these  final 
course  marks  is  used  for  diploma  purposes  and  in  the  calculation  of  the 
average  for  the  Award  of  Excellence. 
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CHAPTER  2 


EXAMINATION  DESIGN,  DEVELOPMENT,  AND  DESCRIPTION 


Examination  Design  and  Development 

There  were  four  stages  in  the  development  of  the  English  33  Diploma 
Examination:  preparation  of  curriculum  and  examination  specifications, 

examination  design,  development  and  field-testing  of  questions,  and  selection 
of  questions  for  the  final  examination. 

The  Curriculum  Branch  of  Alberta  Education  prepared  curriculum  specifications 
that  identify  the  major  content  areas,  the  specific  objectives  within  each 
area,  and  the  emphasis  each  is  to  receive  in  the  classroom.  The  curriculum 
specifications  were  distributed  to  all  school  jurisdictions  in  the  province  in 
the  publication  Diploma  Examinations : Curriculum  Specifications  for  English 
33  (July  1983).  The  Student  Evaluation  Branch  of  Alberta  Education  selected, 
from  the  prepared  curriculum  specifications,  a set  of  diploma  examination 
specifications  that  formed  the  basis  of  evaluation  given  the  limitations  of  a 
paper-and-pencil  examination  design.  These  diploma  examination  specifications 
constitute  Section  C of  the  Curriculum  Specifications  for  English  33. 

The  examination  design,  complete  with  blueprints,  scoring  guides,  and  sample 
questions,  was  distributed  to  all  school  jurisdictions  in  the  province  in  the 
Student  Evaluation  Branch  publication  Grade  12  Diploma  Examinations  Program: 
English  33  (SEB  Bulletin  Volume  3,  Number  2,  September  1983). 

Under  the  supervision  of  the  Student  Evaluation  Branch,  examination  questions 
were  developed  by  English  33  teachers  from  all  parts  of  the  province.  These 
questions  were  then  field-tested  and  subsequently  revised.  Questions  were 
designed  to  reflect  the  reading,  writing,  and  viewing  objectives  outlined  in 
the  diploma  examination  specifications. 

The  final  examination  was  constructed  from  those  questions  that  best  reflected 
curricular  intent  and  examination  design.  It  was  reviewed  by  a committee  of 
classroom  teachers  who  estimated  difficulty  levels  for  each  question  and 
suggested  revisions.  The  target  mean  for  the  English  33  Diploma  Examination 
was  set  at  65%.  The  final  draft  was  approved  by  an  Examination  Review 
Committee  consisting  of  representatives  from  several  stakeholder  groups  (two 
Grade  12  English  teachers  representing  The  Alberta  Teachers'  Association;  one 
member  each  from  the  Conference  of  Alberta  School  Superintendents;  the 
Universities  Co-ordinating  Council;  and  the  Council  of  Presidents,  Public 
Colleges  of  Alberta)  and  representatives  from  the  Curriculum  Branch,  the 
Student  Evaluation  Branch,  and  the  Regional  Offices  of  Alberta  Education.  The 
Examination  Review  Committee  checked  the  draft  for  content  validity,  accuracy, 
and  technical  merit.  Changes  were  made  to  the  examination  as  recommended  by 
the  committee. 
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Examination  Description 


The  English  33  Diploma  Examination  consisted  of  two  parts:  Part  A:  Written 

Response,  worth  50%  of  the  total  diploma  examination  score,  and  Part  B: 

Reading  (Multiple  Choice),  also  worth  50%  of  the  total  score. 

Part  A:  Written  Response 

Part  A:  Written  Response  consisted  of  three  sections.  Section  I:  Personal 
Response  to  Literature  required  the  student  to  read  a short  selection  that 
served  as  a stimulus  for  a personal  or  reflective  response.  It  was  worth  50% 
of  the  mark  awarded  for  Part  A:  Written  Response,  or  25%  of  the  total  mark  for 
the  English  33  Diploma  Examination.  Section  II:  Functional  Writing  required 

the  student  to  write  a letter  of  application  for  a full-time  job  and  to 

convince  a prospective  employer  to  grant  an  interview.  This  section  was  worth 
30%  of  the  mark  awarded  for  Part  A,  or  15%  of  the  total  mark  for  the 
examination.  Section  III:  Critical  Response  to  Visual  Communication  required 
the  student  to  answer  three  questions  about  a cartoon.  The  student's  answers 
were  required  to  be  in  paragraph  form.  This  section  was  worth  20%  of  the  mark 

awarded  for  Part  A,  or  10%  of  the  total  mark  for  the  examination. 

Students  were  allowed  to  use  a dictionary  and  a thesaurus.  In  sections  I and 
II,  suggestions  for  writing  were  given  and  space  was  provided  for  planning 
activities  and  for  rough  work.  Students  were  allotted  two  and  one-half  hours 
to  write  Part  A:  Written  Response. 


Reporting  Categories  for  Part  A:  Written  Response 

To  provide  meaningful  information  about  student  writing,  it  is  necessary  to 
examine  students'  responses  in  terms  of  writing  components  or  factors  that  can 
be  used  as  reporting  categories.  Factors  evaluated  in  Section  I:  Personal 
Response  to  Literature  were:  Thought  and  Detail  - the  content  present  in  the 

student's  writing,  i.e.,  the  concepts  and  ideas  presented  and  specific  detail 
selected  to  support  these  ideas;  Organization  - the  coherent  development  of  a 
unified  theme  or  thesis  through  an  appropriate  method  of  organization;  Matters 
of  Choice  - the  style  of  writing  used  by  the  student,  choices  of  words  and 
syntactic  structures  used  to  give  emphasis  and  effectiveness  to  writing;  and 
Matters  of  Convention  - the  mechanics  of  writing,  i.e.,  proper  spelling, 
punctuation,  and  grammar.  Factors  evaluated  in  Section  II:  Functional  Writing 
were  Thought  and  Detail,  Matters  of  Choice,  and  Matters  of  Convention.  Those 
evaluated  in  Section  III:  Critical  Response  to  Visual  Communication  were 
Thought  and  Detail  and  Matters  of  Convention.  The  design  of  Part  A:  Written 
Response  is  given  in  Table  1 on  page  5. 
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Table  1 

English  33  Diploma  Examination  Blueprint 
Part  A:  Written  Response 


Part  B:  Reading  (Multiple  Choice) 

Part  B:  Reading  (Multiple  Choice)  consisted  of  65  questions  based  on  8 reading 
selections  or  sets  of  information.*  Information  about  the  numbers  and  types 
of  stimuli  in  Part  B is  presented  in  Table  2. 

Table  2 


Distribution  of  Passage  or  Stimulus  Type 


Passage  or  Stimulus 
Type 

Number  of  Passages  or 
Sets  of  Information 

Number  of 
Questions 

Modern  Drama 

1 

13 

Fiction 

2 

14 

Non-fiction 

1 

11 

Poetry 

2 

14 

Information  Sets 

2 

13 

Total 

8 

65 

Students  were  allotted  two  hours  to  write  Part  B.  They  were  not  allowed  to 
use  a dictionary  or  a thesaurus. 

Reporting  Categories  for  Part  B:  Reading  (Multiple  Choice) 

Questions  were  grouped  into  four  reporting  categories  or  subtests:  Meanings 
(23  questions).  Relationships  of  Form  and  Content  (12  questions).  Human 
Experience  and  Values  (17  questions)  and  Life  Skills  (13  questions).  Each 
reporting  category  required  a minimum  of  six  items  to  achieve  statistical 
reliability.  Table  3 on  page  7 shows  which  items  made  up  each  of  the 
reporting  categories.  In  addition.  Table  3 gives  the  proportion  of  the  total 
examination  mark  assigned  to  each  reporting  category  or  subtest. 

Cognitive  Levels  for  Part  B:  Reading  (Multiple  Choice) 

A further  design  consideration  affecting  the  development  of  Part  B was 
cognitive  level.  Questions  were  classified  according  to  three  cognitive 
levels:  Literal  Understanding  (14  questions).  Inference  and  Application  (43 

questions),  and  Evaluation  (8  questions).  By  considering  cognitive  level  when 
developing  an  examination,  the  Student  Evaluation  Branch  attempts  to  ensure 
that  a variety  of  mental  activities  will  be  used  by  students  as  they  write  the 
examination.  Questions  listed  under  Literal  Understanding  are  expected  to  be 
answered  using  skills  of  recognition  and  recall;  those  listed  under  Inference 
and  Application  are  expected  to  elicit  skills  of  analysis,  interpretation, 
synthesis,  and  application  of  knowledge;  and  questions  listed  under  Evaluation 
are  expected  to  draw  forth  judgmental  skills. 

Table  3 shows  how  many  questions  from  each  reporting  category  have  been  placed 
within  each  cognitive  level.  In  addition.  Table  3 gives  the  proportion  of  the 
total  examination  mark  assigned  to  each  cognitive  level. 


*Sets  of  information  included  an  announcement  and  a letter  written  in  response 
to  it,  and  information  associated  with  moving  to  a new  town. 
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COGNITIVE  LEVEL 

REPORTING  CROSS-REFERENCE  TO  PROPORTION 

CATEGORY  ENGLISH  33  STATEMENT  OF  TOTAL 

OF  CONTENT**  LITERAL  INFERENCE  AND  MARK*** 

UNDERSTANDING  APPLICATION  EVALUATION 


CHAPTER  3 


ADMINISTRATION  OF  THE  EXAMINATION 


In  May  1984,  the  Student  Records  and  Computer  Services  Branch  of  Alberta 
Education  received  the  registration  checklists  from  principals  of  all  high 
schools  in  Alberta  with  students  registered  in  Grade  12.  Principals  were 
asked  to  verify  the  registration  of  students  in  diploma  examination  courses 
within  their  schools  and  to  indicate  any  changes  in  registration  that  might 
have  occurred.  These  verified  lists  allowed  Alberta  Education,  through  the 
Student  Evaluation  Branch,  to  estimate  numbers  of  examinations  required  for 
each  diploma  examination  course  for  each  school,  and  to  prepare  answer  sheets 
and  scoring  sheets  with  specific  student  and  school  identification  information. 

Also  in  May  1984,  each  superintendent  and  high  school  principal  was  sent  a 
copy  of  the  document  Guidelines  and  Procedures  for  Administering  the  Grade  12 
Diploma  Examinations : June  1984  Administration.  This  document  included  an 
explanation  of  the  duties  and  responsibilities  of  the  various  parties  involved 
in  administering  the  examinations  (the  school  jurisdiction,  the  school 
principal  or  Chief  Presiding  Examiner,  and  the  teacher  or  Presiding 
Examiner.)  In  addition,  information  about  security  of  the  examinations, 
confidentiality  of  students'  answers,  exceptional  cases,  and  examination  rules 
was  included. 

After  the  examinations  had  been  printed,  they  were  placed  in  school  packages 
by  subject  according  to  the  registration  checklist  that  had  been  verified  by 
the  principal.  Each  school  was  sent  an  extra  number  of  examinations  — 
approximately  10%  more  than  the  list  indicated.*  This  provided  for  changes  in 
registration  and  for  mature  students. 

A package  of  the  diploma  examinations  was  also  prepared  for  each  Chief 
Presiding  Examiner  (high  school  principal).  In  addition,  information  kits  for 
the  Chief  Presiding  Examiners  (one  per  school)  and  the  Presiding  Examiners 
(approximately  one  for  every  30  students  in  the  school)  were  prepared.  These 
information  kits  contained  specific  instructions  necessary  to  ensure  reliable 
administration  of  the  examinations.  All  of  this  material  was  sent  to  the 
central  office  of  each  school  jurisdiction  in  Alberta  in  early  June  1984  for 
dissemination  to  schools. 


*A11  high  schools  with  students  registered  in  Grade  12  received  a minimum  of 
three  extra  copies  of  each  diploma  examination  to  be  kept  on  file  in  the 
school  as  information  for  teachers  and  students. 
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School  jurisdictions  were  responsible  for  delivering  examination  materials  to 
schools  before  the  administration  dates.  School  jurisdictions  were  also 
responsible  for  keeping  the  examinations  secure  prior  to  administration  and 
for  supervising  the  collection  of  examination  materials  from  schools  after 
administration. * 

Examinations  were  returned  to  Alberta  Education  in  the  following  manner: 

School  jurisdictions  in  zones  1,  4,  5,  and  6 returned  materials  as  directed  by 
their  Regional  Offices  of  Education;  jurisdictions  in  zones  2 and  3 returned 
materials  directly  to  the  Student  Evaluation  Branch  receiving  area  at  the 
Legislature  Annex,  9718  - 107  Street,  Edmonton.  At  the  Legislature  Annex,  the 
examinations  from  each  school  were  checked  against  the  school's  List  of 
Candidates  and  prepared  for  marking. 


*Grade  12  diploma  examinations  written  by  students  enrolled  in  category  I 
private  schools  were  administered  by  Regional  Office  of  Education  personnel. 


CHAPTER  4 


SCORING  OF  PART  A:  WRITTEN  RESPONSE 


Organization  of  Markers 

A marking  centre  was  established  in  Edmonton  at  the  Legislature  Annex. 

Seventy  teachers  from  across  the  province  of  Alberta  spent  approximately  one 
week  (July  4 to  July  12,  1984)  marking  Part  A:  Written  Response.  To  qualify 
as  a marker,  each  teacher  was  required  to  be  currently  teaching  English  33,  to 
have  a valid  permanent  Alberta  teaching  certificate,  and  to  have  taught 
English  33  for  at  least  two  years. 

The  70  markers  were  divided  into  14  groups  to  facilitate  the  administration  of 
scoring.  Each  group  was  headed  by  a group  leader  who  was  responsible  for 
helping  to  train  the  group  of  markers  and  to  conduct  reliability  reviews  (see 
page  12).  The  14  group  leaders  met  at  the  marking  centre  for  an  initial 
meeting  with  Student  Evaluation  Branch  personnel  on  Tuesday,  July  3,  1984  to 
review  the  scoring  guides  that  had  been  prepared  (see  pages  29  to  35)  and  to 
read,  discuss,  and  reach  a consensus  about  the  quality  of  papers  selected  for 
training  purposes. 


Training 

On  Wednesday,  July  4,  the  70  markers  met  at  the  Legislature  Annex.  Most  of 
this  first  day  was  used  for  training  the  markers.  The  training  began  with  an 
overall  review  of  a marker's  manual,  a copy  of  which  had  been  provided  for 
each  of  the  markers.  The  criterion-referenced  scoring  scales  for  sections  I 
and  II  of  Part  A:  Written  Response  and  the  key  to  Section  III  were  presented 
to  the  markers.  Markers  then  regrouped  into  their  14  small  groups  to  read  and 
discuss  papers  that  had  been  selected  for  training  purposes. 


Marking 

The  remainder  of  the  scoring  time  was  used  for  independent  evaluation  of 
student  papers.  In  addition,  volunteers  remained  for  evening  scoring  sessions 
on  Monday,  July  9,  and  Tuesday,  July  10.  Volunteers  also  remained  for  a 
"clean-up"  session  on  Friday,  July  13,  1984. 

Before  papers  were  made  available  to  the  markers,  student  identification 
information  was  removed,  and  the  papers  were  randomly  sorted  into  bundles  of 
eight.  Bundles  were  grouped  into  seven  batches  of  approximately  800  papers 
each.  The  scoring  and  processing  of  the  papers  in  each  batch  was  completed 
before  a new  batch  was  started.  The  purpose  of  the  batch  system  is  to  allow 
for  papers  receiving  discrepant  scores  to  be  located,  processed,  and  rescored 
during  the  scoring  session. 

Even  though  all  student  identification  information  had  been  removed  from  each 
paper,  markers  were  instructed  to  refer  to  their  group  leader  any  paper  that 
they  suspected  was  written  by  one  of  their  own  students. 
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All  papers  were  scored  independently  three  times.  Markers  took  a bundle  of 
papers  from  a table  labelled  "Unmarked  Papers"  and  checked  the  back  cover  of 
each  paper  in  the  bundle  to  make  certain  they  had  not  yet  scored  any  of  the 
papers.  (Each  marker  had  previously  been  assigned  a number  that  was  entered 
on  the  back  cover  of  each  of  the  papers  as  scored. ) Markers  returned  to  their 
groups  and  independently  scored  each  of  the  papers  in  the  bundle  by  applying 
the  criterion-referenced  scales  and  the  key  shown  in  Appendix  A.  Markers 
assigned  a score  of  0 (Insufficient)  to  5 (Proficient)  on  each  scale  applied 
to  sections  I and  II  of  the  paper.  They  then  assigned  appropriate  scores  for 
content  to  the  three  questions  in  Section  III  and  assigned  a score  for  Matters 
of  Convention  for  all  three  questions  in  Section  III  taken  together.  Markers 
entered  all  of  this  information  on  a machine-scorable  sheet  (see  Appendix  B, 
page  37),  wrote  their  own  marker  number  on  the  back  cover  of  each  paper,  and 
returned  the  bundle  of  papers  to  another  table  labelled  "Marked  Papers."  They 
then  repeated  this  process. 

Reliability  Reviews 

Reliability  of  results  was  of  prime  concern  during  the  marking  sessions. 
Because  of  this,  reliability  review  sessions  were  conducted  regularly  — at 
10:00  a.m.  and  at  2:00  p.m.  July  4 to  July  9,  and  at  10:00  a.m.  July  10  and 
July  11.  Group  membership  changed  during  alternate  reliability  review 
sessions.  At  these  sessions,  all  markers  were  given  identical  papers  to  read 
and  to  evaluate  independently.  The  group  leaders  recorded  these  initial 
scores.  Each  group  then  discussed  the  two  papers  scale  by  scale,  with  the 
discrepant  or  divergent  markers  on  a particular  scale  speaking  first.  Group 
members  were  then  given  the  opportunity  to  alter  their  scores,  and  these 
altered  post-discussion  scores  were  also  recorded  by  the  group  leader.  At  the 
end  of  the  reliability  review,  group  leaders  forwarded  both  sets  of  scores  to 
Student  Evaluation  Branch  personnel,  who  then  tallied  all  scores  and  posted 
the  pre-  and  post-discussion  distribution  of  scores  for  that  session.  This 
information  provided  useful  feedback  for  monitoring  the  application  of  the 
scoring  guides  by  the  markers. 

As  a further  check  on  reliability,  markers  were  given  individual  performance 
reports  (see  Appendix  C,  page  39).  These  performance  reports  gave  information 
on  numbers  of  papers  scored  by  an  individual  and  the  percentage  of  these 
papers  requiring  rescoring  because  of  discrepancy.  The  reports  also  informed 
individual  markers  of  the  percentage  of  scores  on  each  of  the  scale  points  (0 
to  5)  they  were  assigning  for  each  scale.  In  addition,  markers  were  shown  the 
frequency  and  direction  of  each  discrepancy  on  each  scale.  Using  the 
performance  reports,  each  marker  could  compare  his  own  performance  with  that 
of  the  group  of  70  markers  as  a whole. 


Scale-by-Scale  Assignment  of  Final  Mark 

Once  the  papers  from  a batch  had  received  three  readings,  their  scoring  sheets 
(see  Appendix  B,  page  37)  were  scanned  by  computer.  The  computer  was 
programmed  to  assign  the  median  score  for  each  scale  as  the  final  score.  If, 
for  example,  the  first  marker  had  awarded  a particular  paper  a 3 for  Thought 
and  Detail  on  Section  I,  the  second  marker  a 2,  and  the  third  a 4,  the 
computer  would  have  assigned  that  paper  the  median  score  of  3 for  that 
particular  scale.  In  this  case,  the  scores  given  by  the  second  and  third 
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markers  (2  and  4)  would  have  been  considered  divergent,  since  they  each 
differed  from  the  median  score  by  one  scale  point.  If,  on  the  other  hand,  the 
first  marker  had  awarded  a 4,  the  second  a 4,  and  the  third  a 2,  the  computer 
would  have  assigned  the  median  score  of  4 for  the  scale.  In  this  second  case, 
the  scores  given  by  the  first  two  markers  would  have  been  in  agreement 
(both  4’s),  but  the  score  given  by  the  third  marker  (2)  would  have  been 
considered  discrepant,  since  it  differed  from  the  median  score  by  more  than 
one  scale  point. 


Handling  of  Discrepant  Papers 

Discrepancy  on  one  scale  alone  was  not  enough  to  require  a fourth  reading  for 
any  paper.  If,  however,  a number  of  scales  on  a particular  paper  were 
discrepantly  scored  by  the  first  three  markers,  that  paper  was  considered  to 
be  discrepant  and  was  given  a fourth  reading.  Scores  assigned  during  the 
fourth  reading  had  to  fall  within  the  range  of  scores  that  had  been 
established  by  the  first  three  markers.  If,  for  example,  the  first  marker  had 
awarded  the  paper  a 1 on  a particular  scale,  the  second  marker  a 3,  and  the 
third  marker  a 3,  the  fourth  marker  could  award  the  paper  a 1,  a 2,  or  a 3 on 
this  same  scale.  The  fourth  marker  could  not  go  outside  the  set  range  of 
scores  to  assign  the  paper  a 0,  a 4,  or  a 5 on  that  scale. 
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CHAPTER  5 


RESULTS  AND  OBSERVATIONS 


Total  Examination  Results 

The  results  of  the  English  33  Diploma  Examination  are  summarized  in  Table  4. 
The  average  mark  for  all  students  for  the  total  examination  was  63.7%. 
Students  achieved  an  average  of  31.7  marks  out  of  50  on  Part  A:  Written 
Response,  and  an  average  of  32.0  marks  out  of  50  on  Part  B:  Reading  (Multiple 
Choice) . 


Table  4 

Total  Examination  Results 


Mean 

Standard  Deviation* 

Total  Examination 

63.7% 

10.9% 

Part  A:  Written 

Response 

31.7/50 

5.9 

Part  B:  Reading 

(Multiple  Choice) 

32.0/50 

6.6 

N = 6074 


Standard  test  statistics  indicate  that  the  multiple-choice  section  has  a 
satisfactory  level  of  reliability  for  a test  measuring  a broad  range  of 
concepts  and  skills.  The  KR-20**  coefficient  for  the  multiple-choice  portion 
of  the  examination  was  0.833. 


*Standard  Deviation  is  a measure  of  the  variation  in  the  distribution  of 
scores.  In  a normal  distribution,  68%  of  the  scores  will  fall  within  one 
standard  deviation  of  the  mean. 

**The  KR-20  coefficient  is  a measure  of  internal  consistency  ranging  from 
0 to  1. 
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Standard-Setting 


Every  effort  was  made  to  design  an  English  33  diploma  examination  that  would 
be  a valid  and  reliable  measure  of  what  students  can  be  expected  to  know  as  a 
result  of  instruction  in  this  course.  A specific  standard  or  level  of 
expectation  inherent  in  the  examination  was  established  through  careful  test 
development  procedures. 

To  ensure  that  each  form  of  examination  administered  in  1984  would  be 
parallel,  the  Student  Evaluation  Branch  adopted  a process  of  standard-setting 
that  involved  classroom  teachers  in  making  judgments  about  the  difficulty  of 
the  examination. 

Twenty  of  the  markers,  representing  the  various  zones  of  the  province, 
reviewed  the  difficulty  level  of  each  question  in  terms  of  a borderline 
passing  student  (who  merits  50%) . After  these  teachers  gave  their  initial 
judgments  on  question  difficulty,  they  were  given  information  about  the  actual 
distribution  of  students'  examination  marks.  They  were  then  given  the 
opportunity  to  modify  their  judgments.  Following  a more  general  procedure, 
the  standard  setters  also  suggested  achievement  levels  for  the  borderline  B 
and  borderline  A students  (who  merit  65%  and  80%  respectively) . 

For  English  33,  teachers  suggested  that  a slightly  smaller  number  of  students 
should  receive  A's  and  F's  than  in  the  actual  distribution,  and  that  a 
slightly  larger  number  should  receive  C's  and  B's.  However,  teachers' 
suggestions  were  so  close  to  the  actual  distribution  of  marks  in  English  33 
that  it  was  decided  to  accept  the  actual  distribution  as  the  standard. 


Relationship  Between  Examination  Marks  and  School -Awarded  Marks 

The  averages  for  school-awarded  marks,  examination  marks,  and  final  blended 
marks  were  58.0%,  63.7%,  and  61.8%  respectively.  The  percentages  of  students 
receiving  A's,  B's,  C's,  and  F's  are  given  in  Table  5. 


Table  5 

Percentage  Distribution  of  A's,  B's,  C's,  and  F's 


Letter 

Grade 

School-Awarded 

Mark 

Examination 

Mark 

Final  Blended 
Mark* 

A (80-100%) 

2.4 

5.8 

2.1 

B (65-79%) 

27.4 

43.9 

36.1 

C (50-64%) 

53.1 

40.2 

55.8 

F (0-49%) 

17.1 

10.1 

6.0 

*These  final  blended  marks  include  the  marks  of  students  who  did  not  receive 
BOTH  a school -awarded  mark  and  an  examination  mark  for  various  reasons  (see 
pages  1 and  2).  Also,  there  are  fewer  F's  in  this  distribution  because  of 
regression  toward  the  mean.  (For  example,  the  students  who  received  F as  a 
school-awarded  mark  were  not  necessarily  the  same  students  who  received  F as 
an  examination  mark. ) 
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The  correlation  between  examination  marks  and  school-awarded  marks  was  0.409, 
indicating  that  a relatively  weak  relationship  exists  between  these  two  types 
of  marks.  It  is  reasonable , however,  to  expect  differences  between  these  two 
marks  at  the  individual  student  level  because  the  examination  mark  is  limited 
to  reflecting  achievement  only  in  those  objectives  in  English  33  that  can  be 
examined  by  a paper-and-pencil  test,  while  the  school -awarded  mark  reflects 
achievement  in  listening,  speaking,  viewing,  and  all  other  course  objectives. 
The  school-awarded  mark  also  reflects  work  completed  by  the  student. 

Results  for  Part  A:  Written  Response 

The  results  for  Part  A:  Written  Response  are  given  in  tables  6,  7,  and  8.  The 
three  tables  examine,  in  turn,  the  three  different  sections  of  Part  A. 

Section  I:  Personal  Response  to  Literature 

Table  6 


Percentage  Distribution  of  Scores  on  Section  I 


Score 

(Scale  Points) 

Reporting  Category 

Thought  and 
Detail 

Organization 

Matters  of 
Choice 

Matters  of 
Convention 

5 

(Proficient) 

1.7 

1.8 

2.6 

3.5 

4 

(Capable ) 

22.6 

24.7 

23.8 

32.7 

3 

(Adequate ) 

58.5 

63.2 

59.2 

45.2 

2 

(Limited) 

14.9 

8.6 

12.0 

14.8 

1 

(Poor) 

1.7 

1.0 

1.8 

3.1 

0 

(Insufficient) 

0.7 

0.7 

0.7 

0.7 

N = 6074 


Observations  regarding  the  results  for  Section  I:  Personal  Response  to 
Literature  are  as  follows:  For  Thought  and  Detail,  82.8%  of  the  students 

scored  at  an  Adequate  level  or  better  (received  scores  of  3,  4,  or  5),  and 
16.6%  of  the  students  scored  Limited  or  Poor  (received  scores  of  2 or  1).  For 
Organization,  89.7%  of  the  students  scored  Adequate  or  better,  and  9.6%  scored 
Limited  or  Poor.  For  Matters  of  Choice,  85.6%  of  the  students  scored  Adequate 
or  better,  and  13.8%  scored  Limited  or  Poor.  For  Matters  of  Convention,  81.4% 
of  the  students  scored  Adequate  or  better,  and  17.9%  scored  Limited  or  Poor. 
Only  0.7%  of  the  students  did  not  attempt  Section  I,  or  produced  compositions 
that  were  considered  to  be  insufficient  for  marking  purposes  or  were  off-topic. 
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Section  II:  Functional  Writing 


Table  7 

Percentage  Distribution  of  Scores  on  Section  II 


Score 

(Scale  Points) 

Reporting  Category 

Thought  and 
Detail 

Matters  of 
Choice 

Matters  of 
Convention 

5 

(Proficient ) 

1.7 

1.4 

3.9 

4 

(Capable ) 

23.6 

20.3 

33.9 

3 

(Adequate ) 

52.2 

63.6 

46.0 

2 

(Limited) 

21.1 

13.0 

13.6 

1 

(Poor) 

1,0 

1.1 

2.1 

0 

(Insufficient) 

0.5 

0.5 

0.5 

N = 6074 


Observations  about  the  results  for  Section  II  are  as  follows:  For  Thought  and 

Detail,  77.5%  of  the  students  scored  at  an  Adequate  level  or  better  (received 
scores  of  3,  4,  or  5),  and  22.1%  of  the  students  scored  Limited  or  Poor 
(received  scores  of  2 or  1).  For  Matters  of  Choice,  85.3%  of  the  students 
scored  Adequate  or  better,  and  14.1%  scored  Limited  or  Poor.  For  Matters  of 
Convention,  83.8%  of  the  students  scored  Adequate  or  better,  and  15.7%  scored 
Limited  or  Poor.  Only  0.5%  of  the  students  did  not  attempt  Section  II,  or 
produced  compositions  that  were  considered  to  be  insufficient  for  marking 
purposes  or  were  off-topic. 
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Section  III:  Critical  Response  to  Visual  Communication 


Table  8 

Percentage  Distribution  of  Scores  on  Section  III 


Reporting  Category 

Score* 

Thought  and 

Detail 

Writing  Skills 

Question 

#1  Question 

#2  Question  #3 

Score 

(Scale  Points) 

Matters  of 
Convention 

5 

(Proficient ) 

2.8 

4 

31.5 

4 

(Capable) 

30.6 

3 

21.4 

25.2 

51.0 

3 

(Adequate) 

50.9 

2 

36.7 

43.6 

29.7 

2 

(Limited) 

12.2 

1 

7.1 

15.5 

13.2 

1 

(Poor) 

2.5 

0 

3.3 

15.7 

6.1 

0 

( Insufficient ) 

1.0 

N = 6074 


^Maximum  score  for  question  #1  was  4;  maximum  score  for  questions  #2  and  #3 
was  3 . 


Observations  about  the  results  for  Section  III  are  as  follows:  For  Thought 

and  Detail  in  question  #1,  31.5%  of  the  students  scored  full  marks  (4  out  of 

4) ,  21.4%  scored  3 marks,  36.7%  scored  2 marks,  7.1%  scored  1 mark,  and  3.3% 
of  the  students  received  no  marks.  For  Thought  and  Detail  in  question  #2, 
25.2%  of  the  students  scored  full  marks  (3  out  of  3),  43.6%  scored  2 marks, 
15.5%  scored  1 mark,  and  15.7%  of  the  students  received  no  marks.  For  Thought 
and  Detail  in  question  #3,  51.0%  of  the  students  scored  full  marks  (3  out  of 
3),  29.7%  scored  2 marks,  13.2%  scored  1 mark,  and  6.1%  of  the  students 
received  no  marks.  For  Writing  Skills  (Matters  of  Convention)  in  Section  III, 
84.3%  of  the  students  scored  Adequate  or  better  (received  scores  of  3,  4,  or 

5) ,  14.7%  of  the  students  scored  Limited  or  Poor  (received  scores  of  2 or  1), 
and  1.0%  of  the  students  produced  writing  considered  to  be  insufficient  for 
marking  purposes. 
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Results  for  Part  B:  Reading  (Multiple  Choice) 


The  results  for  Part  B:  Reading  (Multiple  Choice)  by  reporting  category  are 
presented  in  Table  9.  It  is  important  to  note  that  one  question  (#48)  from 
the  first  reporting  category.  Meanings,  was  deleted  from  Part  B.  This  meant 
that  the  50%  of  the  total  mark  for  the  English  33  Diploma  Examination  assigned 
to  Part  B was  calculated  from  a possible  maximum  raw  score  of  64.  Discussion 
of  question  48,  and  of  other  selected  questions,  follows  later  in  this  report. 

Table  9 

Results  for  Part  B:  Reading  (Multiple  Choice) 

(Raw  Scores) 


Reporting  Category 

Mean 

Standard 

Deviation 

1. 

Meanings  (22  questions) 

13.2 

3.5 

2. 

Relationships  of  Form  and  Content  (12  questions) 

7.6 

2.1 

3. 

Human  Experience  and  Values  (17  questions) 

10.0 

2.9 

4. 

Life  Skills  (13  questions) 

9.7 

1.8 

N = 6074 


For  Meanings,  all  students  scored  an  average  of  13.2  questions  correct  out  of 
a possible  total  of  22.  For  Relationships  of  Form  and  Content,  students 
averaged  7.6  questions  correct  out  of  a possible  total  of  12.  For  Human 
Experience  and  Values,  students  averaged  10.0  questions  out  of  17,  and  for 
Life  Skills,  students  averaged  9.7  questions  out  of  13. 

Although  performance  in  the  different  reporting  categories  shows  some 
variation,  these  scores  are  not  directly  comparable.  The  sets  of  questions 
that  make  up  each  reporting  category  were  not  selected  to  be  equal  in  average 
level  of  difficulty,  therefore  differences  may  be  due  to  variations  in 
question  difficulty  rather  than  in  student  performance.  In  combination  with 
jurisdiction  results,  however,  the  means  for  each  reporting  category  can  be 
used  to  detect  patterns  of  relative  strength  or  weakness  in  achievement. 
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Question  response  frequencies  for  all  65  questions  appearing  on  Part  B: 
Reading  (Multiple  Choice)  are  presented  in  Table  10. 

Table  10 


Question  Response  Frequencies 


Question 
Number  Key 

Distribution  of 
Responses  in  % 

Question 

Number 

Key 

Distribution  of 
Responses  in  X 

A 

B 

C 

D 

A 

B 

C 

D 

1 

D 

3 

1 

1 

95 

34 

B 

8 

80 

4 

8 

2 

B 

1 

88 

9 

2 

35 

D 

14 

26 

6 

54 

3 

C 

7 

25 

56 

13 

36 

C 

6 

18 

72 

4 

4 

A 

88 

3 

5 

3 

37 

C 

10 

8 

61 

21 

5 

B 

20 

60 

7 

13 

38 

D 

16 

10 

8 

66 

6 

B 

12 

35 

28 

24 

39 

D 

31 

10 

8 

51 

7 

C 

8 

36 

53 

3 

40 

A 

92 

1 

5 

2 

8 

D 

21 

6 

11 

62 

41 

B 

21 

58 

8 

13 

9 

A 

77 

17 

5 

1 

42 

C 

9 

12 

56 

23 

10 

D 

7 

4 

5 

84 

43 

D 

23 

9 

9 

59 

11 

D 

56 

6 

10 

27 

44 

A 

72 

10 

7 

11 

12 

B 

3 

80 

8 

9 

45 

D 

18 

11 

31 

40 

13 

A 

49 

32 

3 

16 

46 

B 

6 

31 

35 

28 

14 

A 

57 

4 

10 

29 

47 

A 

52 

5 

24 

19 

15 

B 

33 

57 

4 

6 

48* 

B 

21 

53 

17 

9 

16 

D 

5 

11 

24 

60 

49 

D 

2 

2 

5 

91 

17 

B 

13 

50 

27 

10 

50 

B 

11 

34 

32 

23 

18 

C 

10 

36 

51 

3 

51 

A 

95 

2 

1 

2 

19 

C 

3 

19 

64 

14 

52 

D 

6 

3 

3 

88 

20 

B 

4 

70 

18 

8 

53 

C 

12 

27 

60 

1 

21 

A 

55 

7 

18 

20 

54 

C 

12 

31 

54 

3 

22 

C 

5 

5 

85 

4 

55 

A 

78 

14 

2 

6 

23 

B 

4 

86 

5 

5 

56 

A 

95 

2 

1 

2 

24 

C 

17 

3 

57 

22 

57 

A 

81 

1 

5 

13 

25 

D 

9 

17 

17 

57 

58 

C 

2 

4 

88 

6 

26 

C 

5 

3 

88 

4 

59 

B 

16 

64 

10 

10 

27 

A 

58 

4 

35 

3 

60 

A 

62 

10 

19 

9 

28 

C 

14 

16 

61 

9 

61 

B 

6 

72 

17 

5 

29 

C 

18 

12 

47 

23 

62 

A 

76 

6 

10 

8 

30 

D 

18 

23 

4 

55 

63 

C 

8 

5 

75 

12 

31 

C 

37 

22 

30 

11 

64 

C 

9 

16 

55 

20 

32 

D 

20 

20 

8 

52 

65 

B 

21 

52 

17 

10 

33 

A 

39 

21 

32 

8 

N = 6074 


*Question  48  was  deleted  from  the  examination.  This  meant  that  the  50%  of  the 
total  examination  mark  contributed  by  Part  B:  Reading  (Multiple  Choice)  was 
calculated  out  of  a possible  raw  score  of  64.  The  question  response 
frequencies  for  question  48  are  based  on  an  early  sample  (N  = 2712). 


Discussion  of  Selected  Questions 


Question  5 pertains  to  an  excerpt  from  The  Loved  and  the  Lost  by  Morley 
Callaghan.  It  appears  on  the  examination  as  follows: 

5.  The  writer  makes  his  description  effective  through  the  use  of 

A.  extensive  dialogue 

B.  precise  action  words 

C.  repetition  of  key  phrases 

D.  several  appropriate  examples 

Table  10  shows  that  20%  of  the  students  chose  alternative  A;  60%  chose 
alternative  B,  which  is  the  keyed  answer;  1%  chose  alternative  C;  and  13% 
chose  alternative  D.  This  particular  question  was  of  average  difficulty. 


Question  6,  also  pertaining  to  Callaghan's  The  Loved  and  the  Lost , was 
considerably  more  difficult.  It  appeared  as  follows: 

6.  The  MAIN  subject  of  the  excerpt  is  the 

A.  conduct  of  hockey  players 

B.  violence  of  hockey  spectators 

C.  incompetence  of  hockey  officials 

D.  excitement  of  major-league  hockey 

Twelve  per  cent  of  the  students  chose  alternative  A;  and  35%  chose  alternative 
B,  which  is  the  keyed  answer.  Twenty-eight  per  cent  chose  alternative  C,  and 
25%  chose  alternative  D.  This  particular  question  was  intended  to  elicit 
evaluation  skills  and  was  constructed  to  be  a "best  answer"  rather  than  a 
"correct  answer"  type.  Therefore  all  of  the  alternatives  in  question  6 are  to 
some  degree  correct;  all  of  these  aspects  of  hockey  are  dealt  with  in  the 
excerpt.  However,  the  MAIN  focus  of  the  excerpt  is  on  hockey  spectators 
rather  than  on  hockey  players  or  officials,  or  on  the  game  of  hockey  itself. 
For  this  reason  question  6 was  retained  in  spite  of  its  difficulty. 


Question  9 pertained  to  a school  principal's  announcement  and  the  draft  of  a 
letter  written  in  response  to  it.  This  particular  question  asked  the  student 
to  choose  the  most  appropriate  revision  for  a phrase  in  the  draft  of  the 
letter,  and  was  relatively  easy: 

9.  In  revising  paragraph  W to  make  the  language  more  suitable  for  the 

audience,  the  students  should  consider  changing  the  phrase  "real  good" 
to 

A.  important 

B.  admirable 

C.  super  effective 

D.  totally  awesome 
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Seventy-seven  per  cent  of  the  students  chose  alternative  A,  which  is  the  keyed 
answer;  17 % chose  alternative  B;  5%  chose  alternative  C;  and  1 % chose 
alternative  D. 


Question  11,  also  pertaining  to  the  announcement  and  draft,  proved  to  be 
extremely  difficult.  It  required  students  to  re-read  the  text  of  the  draft 
letter  before  answering: 

11.  When  revising  the  letter,  the  students  should  re-order  the  five 

suggestions  so  that  the  suggestion  in  the  number  1 position  is 

A.  "cafeteria  to  close  on  five-minute  notice  if  mess  develops" 

B.  "raise  prices  and  use  revenue  to  hire  extra  cleaning  staff" 

C.  "all  teachers  to  eat  with  students  and  to  supervise" 

D.  "student  supervision" 

Fifty-seven  per  cent  of  the  students  chose  alternative  A;  6%  chose  alternative 
B;  10%  chose  alternative  C;  and  27%  chose  alternative  D,  which  is  the  keyed 
answer.  The  text  of  the  draft  letter  states  that  the  suggestions  are  arranged 
"from  the  one  we  [the  students]  like  the  least  to  the  one  we  like  best."  The 
text  also  states  that  the  student  authors  of  the  letter  hope  that  their 
principal  "will  agree  that  the  idea  about  closing  the  cafeteria  is  the  best, 
and  that  having  students  supervise  is  the  worst."  Clearly  the  suggestion  that 
should  be  in  the  number  1 position  when  the  letter  is  revised  is  alternative  D 
— "student  supervision." 


Question  19  pertained  to  a poem  by  George  Draper  entitled  "Rink  Keeper's 
Sestina."  This  question  was  of  average  difficulty  and  appeared  as  follows: 

19.  The  poet  links  his  ideas  MAINLY  by  using 

A.  rhythm 

B.  flashbacks 

C.  repeated  key  words 

D.  rhetorical  questions 

Three  per  cent  of  the  students  chose  alternative  A;  19%  chose  alternative  B; 
64%  chose  alternative  C,  which  is  the  keyed  answer;  and  14%  chose  alternative 
D.  The  poem  to  which  this  particular  question  pertains  is  a sestina, 
containing  six  stanzas  of  six  lines  each  and  a final  triplet.  The  final  words 
of  each  line,  key  words  in  the  development  of  the  content  of  the  poem,  are 
repeated,  although  in  different  order,  in  each  stanza.  Even  in  the  final 
triplet,  these  six  words  are  repeated  — three  at  the  ends  of  the  lines  and 
one  each  in  the  middle  of  the  three  lines.  In  this  way,  "Esposito"  is  linked 
with  "slapshots,"  "pleasure"  is  linked  with  "hockey,"  "fury"  is  linked  with 
"Hockey,"  and  alternative  C is  indicated  as  the  best  answer  to  question  19. 
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Question  33  pertained  to  an  excerpt  from  A Letter  to  my  Son , by  George 
Ryga.  It  was  relatively  difficult  and  appeared  as  follows: 

33.  The  factor  that  has  been  MOST  important  in  Old  Lepa ' s life  is  his 

A.  love  for  the  land 

B.  need  for  affection 

C.  admiration  of  his  son 

D.  relationship  with  his  sister 

Thirty-nine  per  cent  of  the  students  chose  alternative  A,  which  is  the  keyed 
answer;  21%  chose  alternative  B;  32%  chose  alternative  C;  and  8%  chose 
alternative  D.  The  text  of  the  excerpt  indicates  that  Old  Lepa  has  a 
considerable  affection  for  the  land.  The  major  conflict  implied  in  the 
excerpt  is  between  Old  Lepa's  desire  to  remain  on  the  land  and  his  son 
Stephen's  desire  to  have  him  move  to  town.  Old  Lepa  himself  demonstrates  his 
love  for  the  land  in  lines  43  to  45  when  he  says  "I  should  tell  him  maybe  how 
the  fields  look  in  the  setting  sun. . .black  trees  holding  up  the  sky,  and 
between  them  and  me,  all  them  fields  of  yellow  wheat  glowing  in  a holy  fire!" 
And  when  the  recorded  voice  of  Stephen  states  "Look  — you've  sold  the  land, 
so  why  hang  on  to  this  house  and  garden?  Where  are  your  neighbors  now?  In 
town  — in  apartments  and  rest  homes!"  (lines  82-84),  Old  Lepa  replies  "My  boy 
always  had  a way  with  words  — especially  when  saying  stupid  things!"  (lines 
85-86).  Old  Lepa  loves  the  land  and  is  frustrated  by  his  son's  failure  to 
understand  this  fact. 


Question  46,  pertaining  to  an  article  by  Richard  A.  Blake  entitled  "Simply 
Super:  A Review  of  Superman , " also  proved  to  be  relatively  difficult.  It 
appeared  on  the  examination  as  follows: 

46.  In  lines  85  to  89,  the  writer  suggests  that  the  Superman  story  is 
popular  because  it 

A.  teaches  the  value  of  innocence 

B.  satisfies  our  need  for  amusement 

C.  reflects  the  cynicism  of  our  age 

D.  challenges  the  reader's  imagination 

Six  per  cent  of  the  students  chose  alternative  A;  31%  chose  alternative  B, 
which  is  the  keyed  answer;  35%  chose  alternative  C;  and  28%  chose  alternative 
D.  Lines  85  to  89  of  the  text  state: 

Jackie  Hudson  knew  the  secret.  He  knew  that  his  comic  book  was  a 
valuable  piece  of  property  and  he  made  his  customers  pay  for  it.  He 
did  not  have  to  speculate  on  America's  search  for  a hero  or  the  age  of 
cynicism  that  trivializes  the  heroic.  No,  he  knew  a good  comic  book 
when  he  saw  one. 

Implied  in  this  paragraph  is  the  idea  that  the  Superman  story  (the  "good  comic 
book")  is  popular  ("a  valuable  piece  of  property")  because  it  is  entertaining 
— it  satisfies  our  need  for  amusement.  Its  owner,  Jackie  Hudson,  is  an 
intelligent  enough  young  entrepreneur  to  recognize  this  fact,  and  does  not 
have  to  speculate  about  any  present  "age  of  cynicism"  or  "search  for  a hero." 
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Question  48,  also  pertaining  to  the  article  by  Richard  A.  Blake,  was  deleted 
from  Part  B:  Reading  (Multiple  Choice)  before  students'  marks  were 
calculated.  Question  48  appeared  on  the  examination  as  follows: 

48.  The  writer's  attitude  toward  the  movie  Superman  is 

A.  sentimental 

B.  favorable 

C.  skeptical 

D.  critical 

Although  this  question  was  only  slightly  more  difficult  than  average,  with  53% 
of  the  students  choosing  the  keyed  answer  (alternative  B)  as  the  correct 
answer,  close  examination  of  the  item  statistics,  and  of  the  question  itself, 
revealed  that  the  question  contained  two  right  answers.  The  following  lines 
from  the  article  indicate  that  both  alternative  A and  alternative  B are 
correct  answers: 

[The  movie]  opens  with  a small-screen  image  of  an  old  comic  book  and  as 
the  pages  turn  and  lumps  of  nostalgia  rise  in  the  throats  of  the 
pre-Beatles  generation,  the  screen  grows  wider  and  the  sound  rises  for 
what  must  be  the  longest  and  loudest  set  of  credits  in  motion  picture 
history.  The  audience  is  whisked  away  from  its  old  comic  book  memories 
through  the  galaxies  in  a journey  comparable  to  2001:  A Space  Odyssey. 

It  turns  out  to  be  a joyride  from  start  to  finish. 

The  writer  states  that  "lumps  of  nostalgia  rise  in  the  throats  of  the 
pre-Beatles  generation"  during  the  initial  part  of  the  movie.  This  suggests 
that  his  attitude  is,  in  part,  "sentimental"  — alternative  A.  He  goes  on  to 
state  that  the  movie  "turns  out  to  be  a joyride  from  start  to  finish."  This 
indicates  that  his  attitude  toward  the  movie  is  "favorable"  — alternative  B. 
Question  48  was  deleted  because  of  the  two  acceptable  answers. 


Question  54  was  another  question  that  proved  to  be  slightly  more  difficult 
than  average.  It  pertained  to  a set  of  materials  on  planning  a move,  and 
appeared  on  the  examination  as  follows: 

54.  Assuming  that  Robin  wants  to  establish  financial  independence,  which 
of  the  following  sections  of  Robin's  budget  would  MOST  PROBABLY  be 
dropped  if  an  emergency  were  to  arise? 

A.  laundry,  medicine,  and  gifts 

B.  money  to  Dad  to  repay  loan 

C.  clothing 

D.  rent 

Twelve  per  cent  of  the  students  chose  alternative  A;  31%  chose  alternative  B; 
54%  chose  alternative  C,  which  is  the  keyed  answer;  and  3%  chose  alternative 
D.  This  question  was  designed  to  elicit  evaluation  skills  and  calls  for  a 
judgment  to  be  made  on  the  part  of  the  student.  The  majority  of  the  students 
determined  that  the  least  essential  element  in  the  budget  would  be  new 
clothing.  Also,  this  question  discriminated  relatively  well,  with  66%  of  the 
high-achieving  group  answering  correctly  and  only  41%  of  the  low-achieving 
group  answering  correctly. 
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Some  of  the  questions  on  Part  B were  easy,  with  more  than  90 % of  the  students 
answering  correctly.  These  questions  were  retained  because  they  discriminated 
appropriately  between  high  and  low  achievers,  and  because  they  have  curricular 
validity.  Question  56  is  a typical  representative  of  this  group  of  easy 
questions.*  It  pertained  also  to  the  set  of  materials  on  planning  a move,  and 
appeared  on  the  examination  as  follows: 

56.  Sandy  and  Tracy  will  be  a help  to  Robin  because  they 

A.  know  Nalwen  well 

B.  have  an  automobile 

C.  are  expert  swimmers 

D.  can  lend  Robin  money 

Ninety-five  per  cent  of  the  students  chose  alternative  A,  which  is  the  keyed 
answer;  2 % chose  alternative  B;  1 X chose  alternative  C;  and  2 % chose 
alternative  D.  In  order  to  answer  question  56,  students  had  to  reread  one  of 
the  pieces  of  information  provided. 


*0ther  questions  answered  correctly  by  more  than  90%  of  the  students  were 
questions  1,  40,  49,  and  51. 


Summary  of  Observations 
Total  Examination 

Achievement  on  the  English  33  Diploma  Examination  was  generally  regarded  as 
acceptable  by  teachers  involved  in  the  examination  development  and  marking 
processes.  The  examination  mean  was  63.7%.  The  fact  that  only  10.1%  of  the 
students  writing  the  examination  scored  less  than  50%  suggests  that  students 
are  successfully  learning  those  concepts  and  skills  covered  by  the  English  33 
Diploma  Examination.  It  should  be  noted,  however,  that  there  were  few  high 
achievers  on  this  particular  examination.  Only  5.8%  of  the  students  scored 
80%  or  better. 


Part  A:  Written  Response 

Students  responded  well  to  the  writing  assignments,  often  with  fresh  and 
creative  ideas.  On  Section  I:  Personal  Response  to  Literature,  the  vast 
majority  of  students  (81.4%  to  89.7%)  scored  Adequate  or  better  depending  upon 
the  reporting  category  being  examined.  On  Section  II:  Functional  Writing, 
again  the  majority  of  students  (77.5%  to  85.3%)  scored  Adequate  or  better 
depending  upon  the  reporting  category  being  examined.  On  Section  III: 

Critical  Response  to  Literature,  students  scored  well  on  Thought  and  Detail, 
with  25.2%  to  51.0%  receiving  full  marks  depending  on  the  question  being 
addressed.  Students  also  scored  well  on  Writing  Skills  (Matters  of 
Convention),  with  84.3%  scoring  Adequate  or  better.  Overall,  students 
performed  about  as  well  as  was  expected  on  the  written  section  of  the  English 
33  Diploma  Examination. 


Part  B:  Reading  (Multiple  Choice) 

Achievement  on  Part  B:  Reading  (Multiple  Choice)  was  also  about  as  well  as  was 
expected.  The  overall  mean  on  Part  B:  Reading  (Multiple  Choice)  was  32.0 
marks  out  of  50.  This  is  an  acceptable  standard  of  achievement. 
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APPENDIX  A 

ENGLISH  33  WRITTEN-RESPONSE  SCORING  GUIDE 


Scoring  Guide  for  Section  I:  Personal  Response  to  Literature 

Thought  and  Detail  (Curriculum  Concepts  1,,  3,  4,  6,  7,  8,  9,  12) 

5 PROFICIENT  writing  presents  a situation  that  has  been  effectively 
established.  Precise  details  about  actions  and/or  characters  are 
deliberately  chosen  and  add  clarity.  Retrospection  is  insightful  and 
mature . 

4 CAPABLE  writing  presents  a situation  that  has  been  clearly  and 

appropriately  established.  Specific  details  about  actions  and/or 
characters  are  well-defined  and  plausible.  Retrospection  is  perceptive. 

3 ADEQUATE  writing  presents  a situation  that  has  been  clearly  established. 
Details  about  actions  and/or  characters  are  clear  but  tend  to  be 
general.  Retrospection  is  conventional  but  is  clearly  stated  or  clearly 
implied. 

2 LIMITED  writing  presents  a situation  that  has  been  vaguely  established. 
Inappropriate  or  unclear  details  about  actions  and/or  characters  are 
haphazardly  chosen.  Retrospection  is  obscure  or  uncertain. 

1 POOR  writing  presents  an  inappropriate  or  incomprehensible  situation. 
Details  about  actions  and/or  characters  are  unclear,  irrelevant,  or 
absent.  Retrospection  is  not  present  or  is  confusing. 

0 INSUFFICIENT  writing  is  writing  in  which  no  attempt  has  been  made  to 

respond  to  the  assignment  as  stated,  or  is  writing  that  is  so  deficient 
in  length  that  it  is  not  possible  to  assess  thought  and  detail. 


Organization  (Curriculum  Concepts  2,  3,  4) 

5 PROFICIENT  writing  presents  an  introduction  designed  to  promote  further 
reading.  The  middle  is  developed  so  that  sentences  flow  smoothly  and 
coherently  to  an  appropriate  and  effective  conclusion. 

4 CAPABLE  writing  presents  an  effective  introduction.  The  middle  is 

developed  so  that  sentences  are  coherently  related.  The  conclusion  is 
appropriate . 

3 ADEQUATE  writing  presents  a functional  introduction,  a middle,  and  a 

conclusion.  Sentences  are  generally  related,  but  coherence  falters  on 
occasion. 

2 LIMITED  writing  lacks  either  an  introduction  or  a conclusion. 
Relationships  among  sentences  are  frequently  unclear. 

1 POOR  writing  lacks  an  introduction,  a conclusion,  and  coherence. 

0 INSUFFICIENT  writing  is  writing  that  has  been  awarded  a zero  for  thought 

and  detail. 


Continued 
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Scoring  Guide  for  Section  I (continued) 

Matters  of  Choice  (Curriculum  Concept  3) 

5 PROFICIENT  writing  presents  a selection  and  use  of  words  and  structures 
that  is  accurate  and  generally  effective.  Words  are  correctly  used,  and 
many  sentences  have  been  deliberately  structured  for  consistent  effect. 

4 CAPABLE  writing  presents  a selection  and  use  of  words  and  structures  that 
is  generally  accurate  and  frequently  effective.  Words  are  seldom  misused, 
and,  on  occasion,  a sentence  has  been  deliberately  structured  for  effect. 

3 ADEQUATE  writing  presents  a selection  and  use  of  words  and  structures  that 
is  generally  accurate  and  occasionally  effective.  Most  words  and 
structures  are  used  correctly,  but  attempts  to  structure  sentences  for 
effect  are  rare. 

2 LIMITED  writing  presents  a selection  and  use  of  words  and  structures  that 

is  frequently  inaccurate  and  ineffective.  Many  words  and  structures  are 

misused.  No  attempt  is  made  to  use  sentence  structure  for  effect. 

1 POOR  writing  presents  a selection  and  use  of  words  and  structures  that  is 

generally  inaccurate  and  ineffective.  Words  and  structures  are  misused  to 
such  an  extent  that  clarity  suffers. 

0 INSUFFICIENT  writing  is  writing  that  has  been  awarded  a zero  for  thought 
and  detail. 


Matters  of  Convention  (Curriculum  Concept  3) 

5 PROFICIENT  writing  is  essentially  free  from  errors  of  spelling, 

punctuation,  and  grammar.  Errors  that  are  present  do  not  reduce  the 
communicative  power  of  the  composition. 

4 CAPABLE  writing  has  occasional  errors  of  spelling,  punctuation,  and 
grammar.  These  errors  seldom  impede  the  communicative  power  of  the 
composition. 

3 ADEQUATE  writing  has  several  errors  of  spelling,  punctuation,  and 

grammar.  These  errors  occasionally  impede  the  communicative  power  of  the 
composition. 

2 LIMITED  writing  has  frequent  errors  of  spelling,  punctuation,  and 
grammar.  These  errors  often  impede  the  communicative  power  of  the 
composition. 

1 POOR  writing  has  errors  of  spelling,  punctuation,  and  grammar  that  are 
both  noticeable  and  jarring.  These  errors  severely  impede  the 
communicative  power  of  the  composition. 

0 INSUFFICIENT  writing  is  writing  that  has  been  awarded  a zero  for  thought 
and  detail. 


30 


Scoring  Guide  for  Section  II:  Functional  Writing 

Thought  and  Detail  (Curriculum  Concepts  1,  3,  4,  5) 

5 PROFICIENT  writing  presents  accurate  and  precise  information  enhanced  by 
interesting  and  imaginative  details  that  effectively  fulfil  the  purpose. 

4 CAPABLE  writing  presents  accurate  information  substantiated  by  interesting 
and  well-defined  details  that  efficiently  fulfil  the  purpose. 

3 ADEQUATE  writing  presents  essential  information  supported  by  sufficient 
detail  to  fulfil  the  purpose. 

2 LIMITED  writing  lacks  essential  information.  Supporting  details  are  scant 
and  haphazard  so  that  the  purpose  is  only  partially  fulfilled. 

1 POOR  writing  lacks  essential  information  and  supporting  details.  The 
purpose  is  not  fulfilled. 

0 INSUFFICIENT  writing  is  writing  in  which  no  attempt  has  been  made  to 
respond  to  the  assignment  as  stated,  or  is  writing  that  is  so  deficient  in 
length  that  it  is  not  possible  to  assess  thought  and  detail. 

Writing  Skills: 

Matters  of  Choice  (Curriculum  Concept  2,  3,  4,  5) 

5 PROFICIENT  writing  presents  a selection  and  use  of  words  and  structures 
that  is  accurate  and  generally  effective.  Words  are  correctly  used,  and 
many  sentences  have  been  deliberately  structured  for  consistent  effect. 

4 CAPABLE  writing  presents  a selection  and  use  of  words  and  structures  that 
is  generally  accurate  and  frequently  effective.  Words  are  seldom  misused, 
and,  on  occasion,  a sentence  has  been  deliberately  structured  for  effect. 

3 ADEQUATE  writing  presents  a selection  and  use  of  words  and  structures  that 
is  generally  accurate  and  occasionally  effective.  Most  words  and 
structures  are  used  correctly,  but  attempts  to  structure  sentences  for 
effect  are  rare. 

2 LIMITED  writing  presents  a selection  and  use  of  words  and  structures  that 

is  frequently  inaccurate  and  ineffective.  Many  words  and  structures  are 

misused.  No  attempt  is  made  to  use  sentence  structure  for  effect. 

1 POOR  writing  presents  a selection  and  use  of  words  and  structures  that  is 

generally  inaccurate  and  ineffective.  Words  and  structures  are  misused  to 
such  an  extent  that  clarity  suffers. 

0 INSUFFICIENT  writing  is  writing  that  has  been  awarded  a zero  for  thought 
and  detail. 
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Scoring  Guide  for  Section  II  (continued) 


Matters  of  Convention  (Curriculum  Concept  3,  4,  5) 

5 PROFICIENT  writing  is  essentially  free  from  errors  of  spelling, 

punctuation,  and  grammar.  Errors  that  are  present  do  not  reduce  the 
communicative  power  of  the  composition. 

4 CAPABLE  writing  has  occasional  errors  of  spelling,  punctuation,  and 
grammar.  These  errors  seldom  impede  the  communicative  power  of  the 
composition. 

3 ADEQUATE  writing  has  several  errors  of  spelling,  punctuation,  and 

grammar.  These  errors  occasionally  impede  the  communicative  power  of  the 
composition. 

2 LIMITED  writing  has  frequent  errors  of  spelling,  punctuation,  and 
grammar.  These  errors  often  impede  the  communicative  power  of  the 
composition. 

1 POOR  writing  has  errors  of  spelling,  punctuation,  and  grammar  that  are 
both  noticeable  and  jarring.  These  errors  severely  impede  the 
communicative  power  of  the  composition. 

0 INSUFFICIENT  writing  is  writing  that  has  been  awarded  a zero  for  thought 
and  detail. 
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Scoring  Guide  for  Section  III:  Critical  Response  to  Visual  Communication 


(Curriculum  Concepts  1,  3,  4,  5,  13,  14,  16,  17) 


STUDENTS  MAY  PRESENT  IDEAS  NOT  SUGGESTED  BY  THE  EXAMPLES  GIVEN  IN  THE  SCORING 
GUIDE.  THESE  SUGGESTIONS  ARE  NOT  EXCLUSIVE.  EACH  ANSWER  IS  TO  BE  JUDGED  ON 
ITS  MERIT. 

1.  State  TWO  of  the  father's  concerns  as  revealed  by  his  words  and  actions. 
Support  EACH  statement  with  a specific  detail  from  the  cartoon.  Answer  in 
paragraph  form. 

Thought  and  detail:  Four  marks  will  be  awarded  for  this  question:  1 mark 

for  each  of  the  father's  concerns  (2  marks),  and  1 mark  for  each 
supporting  detail  (2  marks). 


Possible  Answers: 


(1)  The  father  is  concerned  with  appearances,  with  conforming  to  the 
status  quo.  He  is  concerned  about  his  family's  standing  in  the 
community.  His  initial  statement  to  his  son  — "the  neighbors  are 
beginning  to  talk"  supports  this  idea.  In  frame  four  of  the  cartoon 
the  father  advises  his  son  not  to  hate  hockey,  implying  that  it  is 
important  to  have  the  same  likes  as  other  members  of  the  community. 

He  is  trying  to  teach  his  son  about  the  "importance"  of  uniformity  in 
the  first  five  frames  of  the  cartoon. 

(2)  The  father  is  also  genuinely  concerned  about  his  son's  development. 

He  wants  his  son  to  live  up  to  his  responsibilities  by  playing  hockey 
and  suggests  that  "part  of  being  grown  up"  is  "learning  to  be  happy 
at  doing  what  you  don't  want  to  do." 

(3)  More  immediately,  the  father  is  concerned  that,  because  of  the 
non-conformist  behavior  of  his  son,  the  family  may  have  to  move  from 
the  neighborhood.  In  the  last  frame  of  the  cartoon  he  tells  his  wife 
"maybe  we  won't  have  to  move"  — but  the  qualification  "maybe" 
suggests  uncertainty.  This  uncertainty  is  also  suggested  by  similar 
qualifications  in  the  sixth  frame  — "I  think  I might  have  brought 
him  around." 


NOTE:  The  depth  of  the  father's  concern  in  all  three  examples  given  above 

is  shown  by  the  father's  kneeling,  by  his  grasping  his  son,  and  by 
his  pleading  expression. 

(4)  The  father  is  also  a husband  and  is  concerned  with  his  wife's 

feelings.  In  the  last  two  frames  of  the  cartoon  he  is  trying  to  be 
hopeful  and  comforting  - "I  think  I might  have  brought  him  around" 
and  "maybe  we  won't  have  to  move."  He  also  places  his  hand  on  his 
wife's  shoulder  in  the  last  frame  of  the  cartoon  - a comforting 
gesture . 


Continued 
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Scoring  Guide  for  Section  III  (continued) 


2.  What  is  the  son's  attitude  toward  his  father?  Refer  to  TWO  specific 

details  in  the  cartoon  that  suggest  this  attitude.  Answer  in  paragraph 
form. 

Thought  and  Detail:  Three  marks  will  be  awarded  for  this  question:  1 

mark  for  identifying  the  son's  attitude,  and  1 mark  for  each  supporting 
detail  (2  marks). 


Possible  answers: 

(1)  The  boy  respects  his  father.  He  remains  facing  him  throughout,  hands 
at  sides  perhaps  suggesting  helplessness.  The  fact  that  he  gives  in 
despite  his  dislike  of  hockey  shows  that  he  cares  for  his  father. 

(2)  The  boy  wishes  to  be  independent  of  his  father.  He  forcefully 

reiterates  his  dislike  of  hockey  (note  larger  letters:  HOCKEY),  and 

looks  irritated  in  frame  3.  His  eventual  agreement  is  made  with 
obvious  reluctance  ("I  ...  I'll  try.  Dad."). 


3.  How  do  you  think  the  mother  feels  about  the  situation  illustrated  in  the 
cartoon?  Refer  to  TWO  details  in  the  cartoon  to  support  your  view. 
Answer  in  paragraph  form. 

Thought  and  Detail:  Three  marks  will  be  awarded  for  this  question:  1 

mark  for  describing  a feeling,  and  1 mark  for  each  supporting  detail  (2 
marks ) . 


Possible  answers: 

(1)  The  mother  shares  the  father's  concerns  about  conformity  and 
appearances  and  about  the  personal  development  of  her  son.  This  is 
shown  in  frame  6 by  her  worried  expression  and  touching  her  husband's 
arm,  as  well  as  by  her  question  "How  did  it  go?" 

(2)  The  mother  is  feeling  hopeful  about  the  situation  illustrated  in  the 
cartoon.  In  frame  7 she  smiles  and  clasps  her  hands,  suggesting  that 
she  is  relieved  about  how  things  have  turned  out. 
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Scoring  Guide  for  Section  III  (continued) 

Writing  Skills  (Matters  of  Convention) 

5 PROFICIENT  writing  is  essentially  free  from  errors  of  spelling, 

punctuation,  and  grammar.  Errors  that  are  present  do  not  reduce  the 
communicative  power  of  the  composition. 

4 CAPABLE  writing  has  occasional  errors  of  spelling,  punctuation,  and 
grammar.  These  errors  seldom  impede  the  communicative  power  of  the 
composition. 

3 ADEQUATE  writing  has  several  errors  of  spelling,  punctuation,  and 

grammar.  These  errors  occasionally  impede  the  communicative  power  of  the 
composition. 

2 LIMITED  writing  has  frequent  errors  of  spelling,  punctuation,  and 
grammar.  These  errors  often  impede  the  communicative  power  of  the 
composition. 

1 POOR  writing  has  errors  of  spelling,  punctuation,  and  grammar  that  are 
both  noticeable  and  jarring.  These  errors  severely  impede  the 
communicative  power  of  the  composition. 

0 INSUFFICIENT  writing  is  so  deficient  in  length  that  it  is  not  possible  to 
assess  matters  of  convention. 
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APPENDIX  B:  ENGLISH  33  SCORING  SHEET 
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